resolver: read package.json exports for a package found through NODE_PATH - #47500

Output of the repro

<gp>/npkg/package.json is {"main":"./legacy.js","exports":{".":"./m.js","./mapped":"./real-target.js","./priv/*":null}}. The child runs with NODE_PATH=<gp> from a folder that has no node_modules.

import() gives the same four answers in Bun. Node does not read NODE_PATH for import.

With two entries, where the first holds a package that has only "exports" and the second holds a copy with "main": Node loads the first copy. Main loads the second copy. This PR loads the first copy.

The rule

<entry>/<name> is the folder of a package when the specifier is <name> or <name>/<subpath>. As in Node, <name> is the whole text in front of the subpath: ver@2.0.0 names a folder ver@2.0.0, and npkg@1.2.3 does not name npkg. If that folder has a package.json whose "exports" is a map, a text or a list, Bun resolves the subpath through it, as the node_modules lookup does. If "exports" has no file for the subpath, the lookup by "main" and by file name runs, as before.

The extension order is that of the place of the folder: the node_modules order inside a node_modules folder, the order of the project outside one. A target that names one file already used that rule. A key that ends in / now uses it too.

Behaviour changes someone will notice

• The "." export wins over "main". In the corpus below, 21 require specifiers and 249 import specifiers resolve to another file. Example: a package shaped like vite 6 ("require": "./index.cjs", "main": "./dist/node/index.js") loads index.cjs. That is the node_modules answer. Node loads the module-sync file.
• An exported subpath resolves: 663 require specifiers and 678 import specifiers that are not found on main.
• A package that has only "exports" is found in the first entry that holds it. With auto-install on, main asked the registry for such a package. Now the copy in NODE_PATH loads and no request goes out.
• The folder of a package comes before a file of its name: with sib.js beside sib/, require("sib") loads from sib/, as in Node.
• The other leniencies of the node_modules lookup apply. pkg/key.js is read as the key "./key". A key that ends in / resolves with the extension and index search. A * target without an extension gets the extension search. Node refuses these. Main finds such a specifier only when a file has the path of the specifier.
• bun build --target=browser reads the "browser" map of the package for a file that "exports" names. A "browser" entry of false for such a file makes the build fail with error: File not found "". The same package in node_modules fails the same way on main.
• A CSS @import of a NODE_PATH package uses the style condition. The target of "imports" (#dep) goes through the same lookup. Node does not read NODE_PATH for either.

Not in this PR: the refusal. A file that "exports" does not name, a subpath that it blocks with null, a package with no require condition and a target that does not exist all still load through the old lookup. In the corpus, 0 specifiers that load on main stop.

Why the refusal waits

• require() of a package shaped like file-type 21 ({"import": ..., "module-sync": ...}, no require condition) loads in Node and loads on main through NODE_PATH. Bun's node_modules lookup refuses it, because Bun has no module-sync condition (module-sync condition support? #20768, draft node: es-module v26 compat — builtin import/require identity, require.cache, import.meta.main, module cache keys, resolver imports/exports (+10 tests) #35697). A refusal through NODE_PATH would stop it there too.
• The refusal is a change of about 20 lines on top of this one, for a later PR. That PR must list four classes that stop: an unexported file, a null target, no matching condition, and a target that cannot load.

What stays as it is

• The error code is MODULE_NOT_FOUND (ERR_MODULE_NOT_FOUND for import).
• "exports": false and "exports": 7 load "main", as in Node. An "exports" object that mixes keys with and without a leading . also goes to the old lookup.
• A subpath longer than 511 bytes goes to the old lookup, because the resolver keeps 512 bytes of a subpath.
• The "browser" map row of esbuild's fixture for Resolution from directories specified via nodePaths ignore package.json exports entries evanw/esbuild#2752 (@scope/pkg3/baz) still does not resolve. No "exports" decides it.
• The node_modules walk still takes npkg@1.2.3 as npkg. Only the NODE_PATH lookup reads the name as Node does.

--aot and a computed specifier

For require("pkg/" + x) the compiler lists the files that the specifier can find and keeps them in the executable. For a package in a NODE_PATH entry it now also tries each "exports" key that names one subpath (pkg/alias). It does that after the walk over the files, which is the same as on main. It lists no folder, so the limit of 1,024 files counts what it counts on main.

Not listed yet: a file that a key with *, or a key that ends in /, has in another folder than the one that the specifier names ("./icons/*": "./dist/icons/*"). bun run loads that file and the executable throws MODULE_NOT_FOUND. Two gap() rows of test/aot/resolver/self-reference-and-node-path.test.ts record it. The walk finds the file when the target has the path of the specifier ("./*": "./*").

The first version of this PR (c857ed3) ran the whole "exports" pass of the node_modules lister for a NODE_PATH package. That pass lists the folder of each * key and counts those files toward the limit. Release builds, resolveComputedSpecifier and bun build --compile --aot:

All is All(TooManyCandidates): the build keeps no file for that site, and the executable throws MODULE_NOT_FOUND.

The lister has the same fault on main for a package in node_modules: with a ./* key and 1,102 files, require("pkg/icons/" + name) builds an executable that throws where bun run loads the file. A list that is narrowed by the prefix repairs both places, but it changes what --aot keeps for node_modules packages. It is a separate change. The two gap() rows can go through it afterwards.

Corpus

test/node_modules/.bun/node_modules: 1,124 packages, 390 with "exports", 3,787 rows (each package, its package.json, and each "exports" key that names one subpath), 3,637 distinct specifiers. Each row holds what require.resolve() and Bun.resolveSync() find. The rows do not hold keys with *, keys that end in /, or files that "exports" does not name.

• node_modules answers changed: 0 of 3,787 (NODE_PATH unset, main against this PR).
• Through NODE_PATH, main differs from its node_modules answer on 946 specifiers. This PR differs on 9. 8 are require() of an entry that has only an import condition, for example estree-walker. 1 is msw/browser, which "exports" blocks with null for Node. All 9 still load by "main" or by file name, as on main.

Measurements

Release builds of main f7607a3 and of this branch on that base (fa8fbbc). Main 34cbc70 is merged into the branch since then. That merge does not touch src/resolver/ or src/sema/resolve/. gdb steps through one warm call of Resolver::load_node_modules and counts instructions, calls of the allocator and system calls. The JIT is off and --no-install is set. Each NODE_PATH entry is a path of 20 bytes.

• node_modules hit: +1 instruction per call (gdb, main f7607a3 vs PR). The count is the same with NODE_PATH set and not used. Inside load_node_modules itself the hit runs the same number of instructions in both builds (453 with "exports", 232 without), and the branch runs one more alignment nop there. The source of the walk is not edited.
• NODE_PATH walk miss: +1,770 to +2,090 instructions per entry that does not answer, +1,540 to +2,110 for the entry that answers, +0 openat per repeated call.
• One exception for openat: a NODE_PATH entry that is a file. One warm call makes 3 openat where main makes 1, because the directory cache does not remember ENOTDIR.
• allocations per call: +0 (node_modules hit), +1 mi_malloc and +1 mi_free (NODE_PATH exports hit, the same as a node_modules exports hit).
• release .text: +3,328 bytes. load_node_modules 13,362 to 13,597 bytes, frame 0x558 in both. load_node_path_package_exports is 1,743 bytes. In the compiler, try_exact_exported_keys is 471 bytes, and list_installed_names has two callers now and is 1,591 bytes out of line (find_candidates 5,868 to 5,152 bytes).
• perf, strace and valgrind are not installed on the machine that took these numbers.

The helper takes the specifier and the subpath by value. Two other shapes cost a node_modules hit 17 to 23 instructions: a &Package argument kept the parsed specifier on the stack, and a second call of Package::parse stopped its inlining into load_node_modules.

Designs that were weighed

• One shared "exports" step for the node_modules walk, NODE_PATH and the auto-install arm. It leaves one copy, but it edits the walk, which runs for every bare specifier. Deduplicate bundler dispatch and output glue, the JS string escaper, and the exports-map retry #31998 folds only the two attempts of the walk.
• A strict step in the NODE_PATH loop, without the leniencies. A package would then answer differently in node_modules and in NODE_PATH.
• The refusal in this PR. See "Why the refusal waits".
• resolver: apply package.json exports to packages found via NODE_PATH #36566 as written. It moved the walk's block out of line, it resolved npkg@1.2.3 to the copy on disk, and it made "exports": false a miss.
• The whole "exports" pass of the --aot lister for a NODE_PATH package (the first version of this PR). See "--aot and a computed specifier".

Other open PRs in the same code

• resolver: treat require.resolve paths entries as search starts only #47184 moves the NODE_PATH loop into load_from_node_path(import_path, kind, out) -> bool. If this PR lands first, that function takes the call of the helper with it.
• resolver: do not retry pkg/x.js as pkg/x after a null "exports" target #46972 changes the .js retry at the walk and at the auto-install arm. The helper has a third copy of that retry. The parity test fails if one copy changes and the helper does not.
• node:module: look up each paths entry of Module._findPath as a directory #47354 makes Module._findPath read "exports" in a NODE_PATH directory. Its description says that _findPath answers false where "exports" has no file. require() through NODE_PATH still loads such a file by the old lookup (see "Why the refusal waits").
• node:module: look up each paths entry of Module._findPath as a directory #47354, resolver: with options.paths, ask the package of the calling module for #imports and its own name #47243, bundler: port esbuild's "browser" map lookup and bundle a disabled path as an empty module #40832 and resolver: do not retry pkg/x.js as pkg/x after a null "exports" target #46972 name resolver: apply package.json exports to packages found via NODE_PATH #36566 as the pending "exports" step for NODE_PATH.

Tests that ran

• Debug build of this branch: test/js/bun/resolve/resolve.test.ts, test/aot/resolver/ (59 pass, 2 todo: the gap() rows), test/bundler/bundler_regressions.test.ts, test/aot/loader/what-steers-the-search.test.ts. On the first commit also test/bundler/esbuild/packagejson.test.ts, test/cli/run/run-autoinstall.test.ts, test/js/node/test/parallel/test-module-globalpaths-nodepath.js and test/js/node/test/parallel/test-require-dot.js. The second commit does not change the resolver.
• cargo test -p bun_sema -p bun_resolver, and the modules tests of bun_aot. The helper in bun_resolver is verified by the JS tests.
• The four new Rust tests are in src/sema/resolve/code/computed/tests.rs. exact_exported_subpaths_of_packages_in_node_path_are_listed and walk_over_node_path_ends_search_before_manifests_are_read fail on main. exports_of_package_in_node_path_list_no_folder and package_in_node_modules_and_in_node_path_counts_once pass on main and fail on c857ed3: they pin that the NODE_PATH arm of the lister counts what it counts on main.
• Release build of main: the three new tests of resolve.test.ts, the three new bundler rows, the AOT pin and the computed-specifier rows fail.